REMARKS 

I. STATUS OF CLAIMS 

Claims 1-51 were examined. 

The applicant cancels claims 2-51 and adds new claims 52-83. 

Claims 1 and 52-83 are now pending. 

Claims 1, 52, 66, and 83 are the independent claims. 

II. THE OBJECTION TO CLAIM 1 

Claim 1 was objected to because there was no antecedent for "said geometry data". In 
response, claim 1 has been amended by deleting that "said". 

m. THE PROVISIONAL NON STATUTORY DOUBLE PATENTING REJECTION 
OF CLAIMS 1-51 

The office action provisionally rejects claims 1-51 for non statutory double patenting over 
claims 1-14 of co-pending application 11/227,074. 

In response, the applicant first notes that claims 2-5 1 have been canceled. Moreover, the 
applicant agrees that the rejection as to claim 1 is provisional and therefore no fiirther response is 
required at this time. 

IV. THE REJECTION UNDER 35 USC 102(B) BASED UPON SCHNEIDER, USP 
6,351,573, OF CLAIMS 1-51 
A. STATEMENT OF THE REJECTIONS 

The office action rejects claims 1-51 under 35 USC 102(b) based upon Schneider, USP 
6,351,573. In support of the rejections, the office action states that: 

Schneider discloses apparatus for combining for combining [sic] first and 
second image data, fi-om two different imaging devices, the first being from an 
ultrasovind detector (col. 1, line 36) and the second being one selected from the 
group of CT, MR, PET, X-ray, and a three dimensional ultrasound imaging device 
(col. 1, lines 34-43). Additionally the second image data is derived from a second 
detector other than the original ultrasound detector (see "one modality" and 
"second modality" of the Abstract). The images are combined with a combination 
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device that is adapted to transfer geometry data that includes at least one of a 
spatial dimension, image position, orientation of the ultrasound image relative to a 
reference point, or information concerning a region or area covered by the images 
(refer to the Abstract in which corresponding and displaying of a second image 
with a line of view established for a first image is taken to be combining of 
images; geometry data is taken to be coordinate data). The apparatus includes 
structure for storing and receiving second image data (see storage and retrieval of 
image data at col. 7, lines 32-34). Schneider further establishes detector 
positioning relative to at least one of a position sensor and a signal source to be 
known within the ultrasound art (see position-indicating probe at col. 1, lines 
60-65 and position sensing articulated arm at col. 2, lines 5 1-53). The invention 
of Schneider includes scaling of the first and second image (see Abstract in which 
"follow image to correspond to the scale, rotation, and position of the lead 
image"). [Office action pages 4 and 5.] 

In response, the applicant discusses Schneider, and then explains how the claims 
distinguish over Schneider. 

B. TEACHINGS OF SCHNEIDER 

Schneider is directed to an imaging device and method. Title. Schneider states his 
concept in the first sentence of the abstract and the first sentence of the Summary of the 
invention, which both read as follows: "a method and apparatus for obtaining and displaying in 
real time an image of an object obtained by one modality such that the image corresponds to a 
line of view estabUshed by another modality." 

Schneider defines the following terms and then uses the defined terms in describing his 
invention:. 

imaging "modality" - "the method or mechanism by which an image is obtained, e.g., 
MRI, CT, video, ultrasound, etc." Column 5 lines 40-42. 

"Lead view. As used herein "lead view" means the line of view toward the object at any 
given time." Column 5 lines 43-50. 

"Lead image. As used herein "lead image" is an image obtained through the same 
modality as the lead view. For example, if the lead view is the physician's view of the surface of 
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the patient, the lead image could be a corresponding video image of the surface of the patient. " 
Column 5 lines 54-58. 

"Follow image. As used herein "follow image" will be an image which must be 
transformed and possibly sliced to the specifications of the lead view and slice depth control. A 
properly sliced and transformed follow image will usually be in a plane parallel with that of the 
lead image, and consequently, orthogonal to the lead view, although other slice contours could be 
used. A properly transformed follow image will be at the same angle of the view as the lead 
image, but at a depth to be separately determined." Column 5 lines 59-67. 

"Fiducial marker. As used herein "fiducial marker" means a feature, image structure, or 
subobject present in lead or follow images that can be used for image analysis, matching, 
coordinate interreferencing or registration of the images and creation of a composite image." 
Column 6 lines 5-9. 

Schneider also defines the terms: Feature extraction; segmentation; classification; 
transformation; and registration. Column 6 lines 10 to column 7 line 4. 

Schneider describes his preferred embodiment referring to Figure 1 , and stating that "A 
lead library 12 and a follow library 14 of images of the object 10 obtained by two different 
modalities communicate with a processing means 16." Column 7 lines 22-24. Schneider's use of 
the past tense "obtained" means the data in these libraries pre exists display thereof The pre 
existence is confirmed by the subsequent statement that "The images (or data gleaned from their 
analysis) are stored within the libraries in an organized and retrievable manner." Column 7 lines 
30-34. 

Schneider teaches that "The processing means 16 interreferences corresponding images in 
image libraries 12 and 14 to provide a map or table relating images or data in one library to 
images or data in the other." Column 7 lines 39-42. 

Schneider teaches that interreferencing occurs after acquisition of the image libraries and 
before actual surgery, stating that " In the preferred embodiment, the lead and follow images are 
interreferenced prior to the svirgical procedure to gather information for use in real time during 
the siargical procedvire. Interreferencing of the lead and follow images gathered in this 
pre-procedure stage is preferably performed by maintaining common physical coordinates 
between the patient and the video camera and between the patient and the MRl device. The first 
step of this preferred method (indicated generally at block 30 of FIG. 2) therefore is to mount the 
patient's head immovably to a holder such as a stereotactic frame. " Column 8 lines 22-32. 

Schneider teaches that the fiducial markers are also necessary to interreference the two 
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image sets: "In the preferred embodiment, object 10 has at least one fiducial marker 22. The 
fiducial marker is either an inherent feature of object 10 (such as a particular bone structure 
within a patient's body) or a natural or artificial subobject attached to or otherwise associated 
with object 10. The system and methods of this invention use one or more fiducial markers to 
interreference the lead and follow images or to interreference lead images acquired in real time to 
lead images or data in the lead image library, as discussed in more detail below." Column 8 
lines 1-9. 

Thus, Schneider teaches that, prior to a surgical procedure, image data is obtained in its 
preferred embodiment using two different modalities, such as video and MRl, specifying their 
relative coordinates of the video camera and MRI to the object to obtain orientation, and 
obtaining relative image scale from fiducial markers. 

Schneider's Fig. 2 clarifies that all of the foregoing steps occur prior to svirgery, by 
describing steps 30-36 as "pre-procedure" steps and steps 38-46 as "real time" steps, and by 
stating that "Information interreferencing the stored lead and follow images is itself stored for use 
for real time imaging during the surgical procedure." 

After describing the acquisition of the two image data sets, Schneider discusses real time 
imaging. Schneider defines "real time imaging" as "images obtained at an interactive rate", 
stating that " the system is ready for use in real time imaging (i.e., images obtained at an 
interactive rate) during a medical procedure." 

Schneider describes the real lime imaging of an object as obtaining a video image of the 
object and orientation data of the video camera, and using fiducial markers to interreference that 
data to the previously obtained, stored, and analyzed video data of the same object, stating: 

After the images stored in the lead and follow libraries have been 
interreferenced, and the fiducial markers in the lead images have been identified, 
the system is ready for use in real time imaging (i.e., images obtained at an 
interactive rate) during a medical procedure. In this example, real time lead 
images of the patient's head along the physician's line of sight are obtained 
through a digital video camera mounted on the physician's head, as in block 38 of 
FIG. 2. Individual video images are obtained via a framegrabber. 

In the preferred embodiment, each video image is correlated in real time 
(i.e., at an interactive rate) with a corresponding image in the lead image library, 
preferably using the digital image analysis techniques discussed above. 
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Specifically, the lead image is segmented, and the subobjects in the segmented 
lead image are classified to identify one or more fiducial markers. Each fiducial 
marker in the real time lead image is matched in position, orientation and size 
with a corresponding fiducial marker in the lead image library and, thus, to a 
corresponding position orientation and size in the follow image library via the 
interreferencing information. 

Thus, Schneider teaches measuring orientation of image data detector and comparing 
previously obtained image data from the same modality to currently obtained image data and 
matching fiducial markers in the two image data sets to interreference in real time image data 
obtained by the same modality. Schneider then teaches that "The follow image is subsequently 
translated, rotated in three dimensions, and scaled to match the specifications of the selected lead 
view." Column 11 lines 8-10. 

Schneider then teaches a first alternative interreferencing using only fiducial markers, 
apparently not using relative orientation data. Column 16 lines 18-38. 

Schneider then teaches a second alternative embodiment in which follow images slices 
are stored prior to surgery. 

Schneider teaches a third alternative embodiment, in which lead images are real time 
interreferenced directly to the stored follow images "by performing the segmentation and 
classification steps between the lead image and the follow images in real time or by using other 
image or pattern matching techniques"; that is by matching orientation and scale using fiducial 
markers. 

In summary, Schneider teaches using fiducial markers to match orientation and scale of 
image sets obtained fi"om different modalities (such as video and MRI). In some embodiments, 

Schneider also discloses transmitting orientation data indicating orientation of different image 
sensors obtaining the two different image data sets. Thus, Schneider discloses transmitting from 
its imaging sensors only image data. Image data is a two or three dimensional matrix in which 
each element of the matrix has associated values. 

It is important to note that Schneider does not rely at all on size information coming with 
the image data. Rather, he identifies the corresponding markers and transforms the size of one of 
the images if the markers in the two images do have the same size. To that end, Schneider 
discloses the segmentation of sub-objects in images in order to match a real time image with a 
stored image (image library). Schneider identifies the fiducial markers in both images. See 
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Schneider, paragraph starting in column 10 at line 64. " Each fiducial marker in the real time 
lead image is matched in position, orientation and size with a corresponding fiducial marker in 
the lead image library and, thus, to a corresponding position orientation and size in the follow 
image library via the interreferencing information. The follow image is subsequently translated, 
rotated in three dimensions, and scaled to match the specifications of the selected lead view. 
Schneider calls this transformation. See column 11, lines 3 et seq. Thus, nothing in Schneider 
discloses or suggests transmitting from the imaging sensors size information. 

C. HOW THIS APPLICATION AND THE CLAIMS DISTINGUISH OVER 
SCHNEIDER 

In contrast to Schneider, this appUcation discloses transmitting additional information 
about the spatial dimension or the orientation fi-om the ultrasound device to the combination 
device, and that information is in addition to the transferred image data. This means that the 
additional information is not pixel values and is information in addition to pixel values. 

In contrast to Schneider, this application discloses transmitting from an ultrasound sensor 
to the image combination device certain additional data (referred to in our specification as 
geometry data) in addition to image data. First paragraph of the SUMMARY OF THE 
INVENTION section. This application discloses using that additional data to control at least one 
of orientation and scale for display of at least the non ultrasound images, such as orientation and 
scale of MRl images. Page 5 of the specification. These differences are defined in the 
independent claims as follows. 

In contrast to Schneider, Claim 1 as amended defines transmitting from the ultrasound 
detector system to the image combination device "geometry data [that] comprise[s] data defining 
... information concerning at least one spatial dimension of an image unit of the first image data". 
The specification defines image unit in terms of units of the image data matrix, stating that 
"geometry data may comprise ... a) information conceming at least one spatial dimension of an 
image vinit of the first image data, in particular of a pixel (preferably separately for different 
directions of the coordinate system). Schneider does not disclose or suggest transmitting from 
the ultrasound detector system to the image combination device information conceming at least 
one spatial dimension of an image unit of the ultrasound data. Therefore, claim 1 is non obvious 
over Schneider. 

In contrast to Schneider, new independent claims 52, 66, and 83 define (quoting from 
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claim 52) "A method for displaying in an image combination device images of an object, 
comprising: ... transferring, from an ultrasound imaging detector ... to said image combination 
device, (1) first image data of said object and (2) additional data; wherein said additional data 
comprises at least one of (a) spatial dimension of an image unit of said first image data and (b) 
orientation of a first image defined by said first image data relative to orientation of said 
ultrasound imaging detector." Schneider does not disclose these limitation. 

As explained for amended claim 1, Schneider does not disclose transmitting "spatial 
dimension of an image unit of said first image data". 

Schneider also does not disclose transmitting "orientation of a first image defined by said 
first image data relative to orientation of said ultrasound imaging detector". Instead, Schneider 
transmits orientation of the image detector to provide relative orientation. Therefore, Schneider 
does not disclose or suggest transmitting "orientation of a first image defined by said first image 
data relative to orientation of said ultrasound imaging detector". Therefore, Schneider does not 
disclose or suggest claim 52. For similar reasons, Schneider does not disclose or suggest 
independent claims 66 and 83. 

Dependent claims 80-83 define limitations relating to the control and transmission of the 
additional data from the ultrasound imaging system to the ultrasound imaging detector and to the 
image combination device. In view of the fact that Schneider is silent with respect to the 
aforementioned additional information, Schneider does not suggest the additional limitations 
defined by these dependent claims. 

V. THE PERSONAL INTERVIEW 

Dioring the personal interview conducted March 21, 2007 between the undersigned. 
Examiner Lauritzen, and Supervisory Patent Examiner (SPE) Mercader, SPE Mercader objected 
to recitation of "geometry data" asserting that claims reciting that phrase read on anything 
relating to geometry of an object. 

In response, the applicant disagrees because both proposed claim 1 and amended claim 1 
presented herein limit "geometry data"to particular kinds of data; amended claim 1 recited 
geometry data is limited to "information concerning at least one spatial dimension of an image 
unit of the first image data." The newly presented claims avoid recitation of "geometry data" but 
contain the related recitation of "additional data" which in turn is limited to "at least one of (a) 
spatial dimension of an image unit of said first image data and (b) orientation of a first image 
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defined by said first image data relative to orientation of said ultrasound imaging detector." 
Accordingly, the applicant believes that SPE Mercader's conclusion regarding the scope of 
claims reciting "geometry data" was incorrect. 

During the personal interview, Examiner Laruitzen and SPE Mercader asserted that the 
claimed "image unit of said first image data" read on fiducial markers. 

In response, the applicant disagrees because that claim interpretation is contrary to what 
the specification discloses. Specifically, the SUMMARY OF THE INVENTION section, first 
paragraph, explains that the invention transmits from the ultrasound imaging system to the image 
combination device both image data and additional data described there as "geometry data." In 
addition, the specification explains that the geometry data includes "a) information concerning at 
least one spatial dimension of an image unit of the first image data, in particular of a pixel 
(preferably separately for different directions of a coordinate system)" (page 5) and states that 
"[s]ince the information concerning at least one spatial dimension of an image unit of the first 
image data (e.g. the pixel size) can be transferred ... the user can choose spatial dimension .. of 
the ultrasound device ....[to obtain] a continuously adjustable pixel size..." (page 6). 

The statement in the specification "an image vinit of the first image data, in particular of a 
pixel" means that a pixel is an example of an image unit of the first image data. A pixel is not a 
property of image data; it is a matrix element of a matrix whose data defines an image. Thus, the 
specification clarifies that image unit of image data is not the image data, but a unit associated 
with the data. Similarly, the fact that this data is described as geometry data, and the 
specification distinguishes geometry data from image data, indicates that "an image unit" does 
not refer to image data. Thus, the examiner's conclusion is contrary to and inconsistent with the 
specification. Therefore, it is unreasonable. 

Finally, during the interview. Examiner Laruitzen and SPE Mercader asserted that 
Schneider inherently or alternatively obviously disclosed transmission of scale data from the 
video detector to the image combination device. 

In response, the applicant disagrees because, as shown in the detailed discussion of 
Schneider herein above, Schneider does not disclose that concept. Moreover, Schneider clearly 
explains that functions by relating orientation and scale of first and second images of the same to 
one another by using common fiducial markers determined pro grammatically from each image 
library. Given that Schneider clearly explains how it interreferences image object sets using 
fiducial markers, there is nothing in Schneider suggesting that its image detectors transmit image 
scale data to Schneider's image combination device. 

15 



In view of the foregoing, the applicants submit that this application is in condition for 
allowance, which is now requested. 

Truly, 



3/26/2007 /RichardNeifeld#35,299/ 

DATE Richard Neifeld 

Reg. No: 35,299 
Attorney of Record 
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